Monitoring system

ABSTRACT

According to one embodiment, a monitoring system includes a processor. The processor accepts first data output from a first detector. The first detector detects a signal caused by equipment. The processor performs a first determination when a first value is in a first state. The first value indicates a state of the first detector or an environment where the equipment is provided. The first determination determines a condition of the equipment by using a first model and the first data. The processor performs a second determination when the first value is in a second state different from the first state. The second determination determines the condition of the equipment by using a second model and the first data. The second model is different from the first model.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No.2018-209910, filed on Nov. 7, 2018; theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to a monitoring system.

BACKGROUND

There is a system that determines the condition of equipment by using amodel. It is desirable to increase the accuracy of the determination bythe system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view illustrating the configuration of amonitoring system according to a first embodiment;

FIG. 2 and FIG. 3 are flowcharts illustrating processing by themonitoring system according to the first embodiment;

FIG. 4 is a schematic view showing the configuration of a monitoringsystem according to a first modification of the first embodiment;

FIG. 5A to FIG. 5D, FIG. 6A, and FIG. 6B are schematic views describingthe processing of the monitoring system according to the firstmodification of the first embodiment;

FIG. 7 is a schematic view illustrating the configuration of amonitoring system according to a second modification of the firstembodiment;

FIG. 8A to FIG. 8C and FIG. 9 are schematic views describing theprocessing of the monitoring system according to the second modificationof the first embodiment;

FIG. 10 is a schematic view illustrating the configuration of amonitoring system according to the second embodiment; and

FIG. 11 is a schematic view illustrating the configuration of amonitoring system according to the embodiment.

DETAILED DESCRIPTION

According to one embodiment, a monitoring system includes a processor.The processor accepts first data output from a first detector. The firstdetector detects a signal caused by equipment. The processor performs afirst determination when a first value is in a first state. The firstvalue indicates a state of the first detector or an environment wherethe equipment is provided. The first determination determines acondition of the equipment by using a first model and the first data.The processor performs a second determination when the first value is ina second state different from the first state. The second determinationdetermines the condition of the equipment by using a second model andthe first data. The second model is different from the first model.

Various embodiments are described below with reference to theaccompanying drawings.

The drawings are schematic and conceptual; and the relationships betweenthe thickness and width of portions, the proportions of sizes amongportions, etc., are not necessarily the same as the actual values. Thedimensions and proportions may be illustrated differently amongdrawings, even for identical portions.

In the specification and drawings, components similar to those describedpreviously or illustrated in an antecedent drawing are marked with likereference numerals, and a detailed description is omitted asappropriate.

FIG. 1 is a schematic view illustrating the configuration of amonitoring system according to a first embodiment.

For example, the monitoring system 100 according to the first embodimentshown in FIG. 1 is used to monitor the condition of equipment.

As shown in FIG. 1, the monitoring system 100 according to the firstembodiment includes a processor 10. In the example of FIG. 1, themonitoring system 100 further includes an outputter 20 and a modelmemory 30. The processor 10 includes, for example, an acceptor 11, acalculator 12, a determiner 13, and a decider 14. The processor 10includes, for example, a processing circuit made of a central processingunit (CPU).

The acceptor 11 accepts first detection data D1 output from a firstdetector, and second detection data D2 output from a second detector.The first detection data D1 is, for example, time-series data. The firstdetection data D1 may be a value. The second detection data D2 is, forexample, a value. The second detection data D2 may be time-series data.Hereinafter, an example is described in which the first detection dataD1 is time-series data, and the second detection data D2 is a value.

The first detector detects a “signal” caused by the equipment. The“signal” includes any measurable physical quantity. For example, the“signal” includes at least one of a sound, a vibration, light, or aradio wave generated from the equipment. The “signal” may include atleast one of a sound, a vibration, light, or a radio wave reflected bythe equipment.

For example, the first detector includes a microphone (mike). Forexample, the equipment which is the monitoring object is a press machineor a rotating machine. The first detector detects the sound when thepress machine presses a processing object, or the sound when therotating machine rotates. The first detector outputs data based on thesound to the processor 10.

The first detector may include a 3-axis acceleration sensor, a vibrationsensor, or an Acoustic Emission (AE) sensor. For example, the firstdetector detects an acceleration or a vibration when the equipmentmoves.

The first detector may include a radio wave sensor. For example, theradio wave sensor radiates a radio wave and detects the reflected wavefrom the equipment. The radio wave sensor is, for example, a dopplersensor and detects the movement of the equipment.

The first detector may include a distance sensor using light or anultrasonic wave. The distance sensor detects the position or themovement of the equipment by measuring the distance between the distancesensor and the equipment.

The first detector may include an image sensor. For example, the imagesensor detects visible light reflected by the equipment. The imagesensor may be included in an imaging device (e.g., a camera). Theimaging device may generate an image based on visible light detected bythe image sensor. The image sensor may detect infrared light. Theimaging device may generate an image based on the infrared lightdetected by the image sensor.

The second detector detects the state of the first detector or theenvironment where the equipment to be monitored is provided. Forexample, the second detector includes a 3-axis acceleration sensormounted to the first detector. The second detector detects at least oneof an orientation, a position, or a movement of the first detector andoutputs, to the processor 10, a value indicating the at least one of theorientation, the position, or the movement.

The second detector may include a temperature sensor, a humidity sensor,or an atmospheric pressure sensor. The second detector detects theenvironment such as the temperature, the humidity, the air pressure,etc., where the equipment to be monitored is provided, and outputs avalue indicating the environment to the processor 10.

The first detection data D1 that is accepted by the acceptor 11 may betransmitted directly to the processor 10 from the first detector. Thefirst detection data D1 may be generated by the data output from thefirst detector being processed by another processor. The seconddetection data D2 that is accepted by the acceptor 11 may be transmitteddirectly to the processor 10 from the second detector. The seconddetection data D2 may be generated by the data output from the seconddetector being processed by another processor.

When accepting the first detection data D1, the acceptor 11 transmitsthe first detection data D1 to the calculator 12. The calculator 12refers to the model memory 30. For example, the model memory 30 stores afirst model. The calculator 12 calculates an evaluation value indicatingthe condition of the equipment by using the first model and the firstdetection data D1.

The first model is, for example, a mathematical model or a physicalmodel. As an example, the first model is a neural network. The firstmodel is trained using data detected by the first detector when theoperation starts or directly after maintenance of the equipment.Typically, the condition of the equipment is good when the operationstarts or directly after maintenance. In other words, the first model istrained using first detection data when the condition of the equipmentis good. For example, the first model is trained using the same data inthe input layer and the output layer. The trained first model functionsas an autoencoder.

Hereinafter, the condition of the equipment when the first model isgenerated is called “good.” The good condition of the equipment also iscalled “normal.”

The calculator 12 inputs the time-series first detection data X (X_(t1),X_(t2), X_(t3), . . . , X_(tn)) output from the first detector into thefirst model and obtains output data Y (Y_(t1), Y_(t2), Y_(t3), . . . ,Y_(tn)) output from the first model. X_(ti) (i=1, 2, 3, . . . , n) isthe value at a time ti included in the first detection data X. Y_(ti) isthe value output when X_(ti) is input to the first model.

If the equipment is normal, output data that is similar to the firstdetection data is obtained from the first model. If the equipment isabnormal, output data that is not similar to the first detection data isobtained from the first model. The calculator 12 calculates the rootmean square of the difference (RMSE) between the first detection dataand the output data. Formula 1 shows the calculation formula of theRMSE. The value that is calculated by the RMSE is used as the evaluationvalue. The evaluation value increases as the condition of the equipmentdegrades compared to the first model when generated.

$\begin{matrix}{{RMSE} = \sqrt{\frac{1}{n}{\sum\limits_{k = 1}^{n}\left( {y_{i -}x_{i}} \right)^{2}}}} & \left\lbrack {{Formula}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Other than the example described above, a recurrent neural network thatincludes an intermediate layer having a Long Short Term Memory (LSTM)structure may be used as the first model. For example, the recurrentneural network is generated as a prediction model of the condition ofthe equipment. The calculator 12 inputs, to the first model, time-seriesdata X1 (X_(t1), X_(t2), X_(t3), . . . , X_(tn)) output from the firstdetector at some time. The calculator 12 obtains output data Y2 (Y_(t2),Y_(t3), Y_(t4), . . . , Y_(tn+1)) output from the first model. Theoutput data Y2 is the predicted value of time-series data X2 (X_(t2),X_(t3), X_(t4), . . . , X_(tn+1)) output from the first detector at asubsequent time. If the condition of the equipment is good, X2 issimilar to the output data Y2. If the condition of the equipment ispoor, the similarity between X2 and the output data Y2 decreases. Thecalculator 12 uses the root mean square of the difference (RMSE) betweenthe first detection data X2 and the output data Y2 as the evaluationvalue.

The calculator 12 transmits the evaluation value to the determiner 13.The determiner 13 determines the condition of the equipment based on theevaluation value. For example, a large evaluation value indicates thatthe condition of the equipment is poor. The determiner 13 compares theevaluation value and a first threshold. When the evaluation value isless than the first threshold, the determiner 13 determines that theequipment is in a first condition.

For example, the first condition corresponds to a good condition. Whenthe evaluation value is not less than the first threshold, thedeterminer 13 determines that the equipment is in a second condition.The second condition indicates that the condition is inferior to thefirst condition. For example, the second condition corresponds to a poorcondition. Hereinafter, the condition of the equipment when theevaluation value is not less than the first threshold is called “poor.”A poor condition of the equipment also is called “abnormal.”

The determiner 13 transmits the determination result to the outputter20. The outputter 20 outputs the determination result in a form that isrecognizable by the user. The outputter 20 includes, for example, atleast one of a monitor, a printer, a speaker, or a light source. Amonitor or a printer displays, to the user, at least one of a character,a symbol, or a numerical value indicating the determination result. Forexample, a speaker emits a tone, music, or a voice indicating thedetermination result. For example, at least one of a light emissionpattern, an intensity, or a color of light that is emitted from a lightsource changes according to the determination result.

When accepting the second detection data D2 transmitted from the seconddetector, the acceptor 11 transmits the second detection data D2 to thedecider 14. Based on the second detection data D2, the decider 14decides whether or not the calculation of the evaluation value using thefirst model is appropriate.

When it is decided that the calculation using the first model isinappropriate, the model that is stored in the model memory 30 isupdated. For example, in the update of the model, a second model that isdifferent from the first model is newly generated. Or, the second modelmay be generated by retraining the first model.

For example, the decider 14 externally transmits the decision resultindicating that the use of the first model is inappropriate. Whenreceiving the decision result, an external calculator generates thesecond model and stores the second model in the model memory 30. Or,when confirming the decision result, the user causes the externalcalculator to generate the second model and store the second model inthe model memory 30.

The second detection data D2 may be transmitted continuously from thesecond detector to the processor 10. For example, the acceptor 11transmits the second detection data D2 to the decider 14 only when thefirst detection data D1 is transmitted to the calculator 12. Accordingto this method, it can be decided whether or not the use of the firstmodel is appropriate at the timing of calculating the evaluation value.

For example, the first model is erased when the second model is storedin the model memory 30. The calculator 12 calculates the evaluationvalue indicating the condition of the equipment by using the firstdetection data D1 and the second model stored in the model memory 30.

Or, after storing the second model in the model memory 30, both thefirst model and the second model may exist in the model memory 30. Forexample, the decision result of the decider 14 is transmitted to thecalculator 12. Based on the decision result of the decider 14, thecalculator 12 selects one of the first model or the second model andcalculates the evaluation value by using the selected model.

Effects of the first embodiment will now be described.

Typically, the following method is employed when determining thecondition of the equipment using some model. First, the model isgenerated using the signal detected from the equipment when thecondition of the equipment is good. Thereafter, an evaluation value iscalculated using this model and the signal detected from the equipment.Because the evaluation value changes according to the condition of theequipment, the condition of the equipment can be determined based on theevaluation value.

On the other hand, the evaluation value changes not only according tothe condition of the equipment but also according to the state of thefirst detector. For example, when the first detector is a microphone,the data that is detected by the microphone changes when the orientationof the microphone changes. If the evaluation value changes due to thestate of the first detector changing, the condition may be determined tobe poor even when the condition of the equipment is good.

For this problem, a method may be considered in which the state of thefirst detector is returned to the original state when the state of thefirst detector has changed. However, the first detector may be mountedin a high location or proximal to dangerous equipment. In such a case,it is not easy to return the state of the first detector to the originalstate. Also, it is not easy to return the first detector accurately tothe original state.

Moreover, the evaluation value changes according to the environmentwhere the equipment is provided. For example, when the equipment isprovided in a space having low airtightness or outside a building, thetemperature, the humidity, or the air pressure of the space where theequipment is provided may change greatly according to changes of theseason, etc. Such changes cause the transmission of sound to themicrophone to change. As a result, the data that is detected by themicrophone changes. It is not easy to calculate the evaluation valuewhile correcting the change of the environment where the equipment isprovided.

There is another method in which the model used to calculate theevaluation value is updated regularly. However, in this method, themodel is updated even when it is unnecessary to update the model.Therefore, the calculation amount to update the model increasesuselessly.

In the monitoring system 100 according to the first embodiment, the datathat is output from the second detector is utilized. For example, thefirst value that indicates the state of the first detector or theenvironment where the equipment is provided is output from the seconddetector. In the monitoring system 100, the model that is used tocalculate the evaluation value is decided according to the first value.

For example, the processor 10 accepts first data output from the firstdetector at some time. The processor 10 performs a first determinationwhen the first value is in a first state. In the first determination,the processor 10 determines the condition of the equipment by using thefirst model and the first data. The processor 10 performs the seconddetermination when the first value is in the second state that isdifferent from the first state. In the second determination, theprocessor 10 determines the condition of the equipment by using thesecond model and the first data. Thereby, the condition of the equipmentcan be determined more accurately even when the state of the firstdetector or the environment where the equipment is provided changes. Thecalculation amount that is necessary for the update of the model can bereduced.

The first state may correspond to a first posture of an attached stateof the first detector. The second state may correspond to a secondposture of the attached state to which the attached state of the firstdetector has changed from the first posture. The second posture isdifferent from the first posture. The posture of the first detector maychange due to drooping by aging of attachment parts, earthquake, ordisassembly for maintenance.

For example, the processor 10 compares the first value to a presetsecond threshold. The first state corresponds to the state in which thefirst value is less than the second threshold. The second statecorresponds to the state in which the first value is not less than thesecond threshold.

For example, the processor may perform a first determination ofdetermining the equipment using a first data and a first model when aposture of an attached state of the first detector is in a first state.The processor may output a result of the first determination. Theprocessor may perform a second determination of determining theequipment using a first data and a second model different from the firstmodel when the posture of the attached state of the first detector is ina second state different from the first state. The processor may outputa result of the second determination.

The object of the monitoring by the monitoring system 100 is, forexample, equipment provided in a manufacturing site, a constructionsite, a power plant, a power substation, an office, a home, a medicalinstitution, etc. By using the monitoring system 100 according to thefirst embodiment, the condition of such equipment can be determined moreaccurately.

The monitoring of a press machine and a rotating machine is described inthe example described above. A solar panel mounted outside a building,etc., may be monitored by the monitoring system 100. For example, thefirst detector includes an infrared sensor. The first detector detectsinfrared light emitted from the solar panel. For example, an image isgenerated based on the infrared light detected by the first detector.The intensity of the infrared light is proportional to the fourth powerof the temperature of the object. Therefore, the image that is based onthe infrared light is greatly affected by the temperature. The imagethat is based on the infrared light also changes when the temperaturechanges according to the season or the weather. The second detectorincludes a temperature sensor. The second detector detects thetemperature (the air temperature) of the location where the solar panelis provided. The monitoring system 100 modifies the model used tocalculate the evaluation value according to the state of the valueindicating the temperature. Thereby, even when the season and theweather change, the condition of the solar panel can be determined moreaccurately.

FIG. 2 and FIG. 3 are flowcharts illustrating processing by themonitoring system according to the first embodiment.

FIG. 2 illustrates the processing when the processor 10 accepts thefirst data output from the first detector. When the acceptor 11 acceptsthe first data (step S1), the calculator 12 refers to the first modelstored in the model memory 30 (step S2). The calculator 12 calculatesthe evaluation value by using the first model and the first data (stepS3). The determiner 13 determines the condition of the equipment basedon the evaluation value (step S4). The outputter 20 outputs thedetermination result (step S5).

FIG. 3 illustrates the processing when the processor 10 accepts thefirst value output from the second detector. When the acceptor 11accepts the first value (step S11), the decider 14 decides whether ornot the first value is in the second state (step S12). When the firstvalue is in the second state, the model is updated by the calculator(step S13). For example, the second model that is generated separatelyfrom the first model is stored in the model memory 30.

The processor 10 may output information indicating the model used todetermine the condition of the equipment. For example, the processor 10outputs first information when the first value is in the first state.The processor 10 outputs second information indicating the second modelwhen the first value is in the second state. Thereby, the user can knowwhich model is used in the determination of the condition of theequipment.

When the model of the model memory 30 is updated, the processor 10 mayoutput information indicating that the model is updated. The user easilycan confirm that the model has been updated. The processor 10 may outputthe first value output from the second detector. For example, the usereasily can confirm that the model has been updated based on the firstvalue.

The processor 10 may output the data output from the first detector.Thereby, for example, the user can determine the condition of theequipment based on the data output from the first detector in additionto the determination by the monitoring system 100.

The processor 10 may output the evaluation value. In other words, theprocessor 10 outputs a first evaluation value calculated using the firstdata and the first model when the first value is in the first state. Theprocessor 10 outputs a second evaluation value calculated using thefirst data and the second model when the first value is in the secondstate. Thereby, for example, the user can know the more specificcondition of the equipment based on the evaluation value.

For example, the decider 14 decides whether or not the use of the firstmodel is appropriate when a new evaluation value is calculated by thecalculator 12. The decider 14 may decide whether or not the use of thefirst model is appropriate by using the new evaluation value in additionto the first value output from the second detector.

For example, when the first value is not less than the second threshold,the decider 14 decides to update the model regardless of the newevaluation value.

When the first value is less than the second threshold, the decider 14decides whether or not the first value is not less than a thirdthreshold. The third threshold is smaller than the second threshold.When the first value is less than the second threshold and not less thanthe third threshold, the decider 14 refers to the new evaluation valueand the evaluation value one-previous. The decider 14 decides to updatethe model when the difference between the new evaluation value and theevaluation value one-previous is not less than a fourth threshold.

According to this processing, the necessity of the update of the modelcan be decided more accurately. By appropriately updating the model, thecondition of the equipment can be determined more accurately.

The processor 10 may accept both data indicating the state of the firstdetector and data indicating the environment where the equipment isprovided. For example, the processor 10 accepts the first valueindicating the state of the first detector, and a second value showingthe environment where the equipment is provided. When the first valueand the second value are in the first state, the processor 10 performsthe first determination determining the condition of the equipment byusing the first model and the first data. When at least one of the firstvalue or the second value is in the second state, the processor 10performs the second determination determining the condition of theequipment by using the second model and the first data. According tothis processing, the condition of the equipment can be determined moreaccurately.

For example, the decider 14 compares the first value to a threshold, andcompares the second value to another threshold. The first statecorresponds to the case where the first value is less than the thresholdand the second value is less than the other threshold. The second statecorresponds to the case where the first value is not less than thethreshold, the case where the second value is not less than the otherthreshold, or the case where the first value is not less than thethreshold and the second value is not less than the other threshold.

FIG. 4 is a schematic view showing the configuration of a monitoringsystem according to a first modification of the first embodiment.

The monitoring system 110 according to the first modification shown inFIG. 4 further includes a first detector 40, a first detection dataprocessor 41, a second detector 50, a second detection data processor51, and a calculator 60.

As described above, the first detector 40 detects a signal caused by theequipment. The first detection data processor 41 processes the datadetected by the first detector 40. The first detection data processor 41transmits the processed first detection data D1 to the processor 10.

As described above, the second detector 50 detects the state of thefirst detector or the environment where the equipment to be monitored isprovided. The second detection data processor 51 processes the datadetected by the second detector 50. The second detection data processor51 transmits the processed second detection data D2 to the processor 10.

For example, the first detection data processor 41 also transmits thefirst detection data D1 to the calculator 60. The calculator 60 updatesthe model based on the decision result transmitted from the decider 14.For example, the decider 14 decides that the first model isinappropriate. When receiving the decision result, the calculator 60updates the model by using the first detection data D1 when the decisionresult is received.

For example, when receiving the decision result, the calculator 60 setsthe first detection data D1 to the input layer and the output layer ofthe model and trains the model. In other words, as an example, thecalculator 60 generates another model that functions as an autoencoder.The calculator 60 stores the generated model in the model memory 30.

For example, the following processing is performed in the monitoringsystem 110.

The processor 10 accepts the first data output from the first detectiondata processor 41. The processor 10 accepts the first value output fromthe second detection data processor 51. The processor 10 performs thefirst determination determining the condition of the equipment by usingthe first model and the first data when the first value is in the firststate. The processor 10 generates the second model by using other data(third data) output from the first detector 40 when the first value isin the second state. After generating the second model, the processor 10performs the second determination determining the condition of theequipment by using the second model and the first data.

The first detector 40, the first detection data processor 41, the seconddetector 50, and the second detection data processor 51 will now bedescribed referring to specific examples.

In one specific example, the first detector 40 is a microphone. Forexample, the first detector 40 detects stationary sound and converts thestationary sound into data (an electrical signal). The stationary soundhas a slight fluctuation of the magnitude and frequency of the sound, orhas no fluctuation.

The first detection data processor 41 includes a pre-processor 41 a, awindowing processor 41 b, and a Fourier transformer 41 c. Thepre-processor 41 a splits the data detected by the first detector 40into frames every K/2 samples. K is an even number. The data that issplit into frames is transmitted to the windowing processor 41 b. Thewindowing processor 41 b multiplies the split data by w(t) which is awindow function. The signal of the input signal y_(n)(t) (t=0, 1, K/2-1)of the nth frame windowed by w(t) is illustrated by Formula 2.

y _(n)(t)=w(t)y _(n)(t)   [Formula 2]

The windowing processor 41 b may perform windowing by overlapping partsof two continuous frames. The overlap length is set to 50% of the framelength. The left side obtained in Formula 3 for t=0, 1, . . . , K/2-1 isused as the output of the windowing processor 41 b.

$\begin{matrix}\left. \begin{matrix}{{y_{n}(t)} = {{w(t)}{y_{n - 1}\left( {t + {K/2}} \right)}}} \\{{y_{n}\left( {t + {K/2}} \right)} = {{w\left( {t + {K/2}} \right)}{y_{n}(t)}}}\end{matrix} \right\} & \left\lbrack {{Formula}\mspace{14mu} 3} \right\rbrack\end{matrix}$

The windowing processor 41 b may use a laterally-symmetric windowfunction for a real signal. An example of windowing by overlapping 50%of two continuous frames will now be described. For example, thewindowing processor 41 b uses the hanning window shown in Formula 3 asw(t).

$\begin{matrix}{{w(t)} = \left\{ \begin{matrix}{0.5 + {0.5{\cos \left( \frac{\pi \left( {t - {K/2}} \right)}{K/2} \right)}}} & {0 \leq t \leq K} \\{0,} & {otherwise}\end{matrix} \right.} & \left\lbrack {{Formula}\mspace{14mu} 4} \right\rbrack\end{matrix}$

Other window functions such as a Hamming window, a Kaiser window, or aBlackman window may be used. The windowed output is transmitted to theFourier transformer 41 c. The Fourier transformer 41 c converts the dataoutput from the windowing processor 41 b into a frequency spectrum. Thefrequency spectrum is separated into an amplitude spectrum and a phasespectrum and transmitted to the processor 10 as the first detection dataD1. A power spectrum may be used instead of the amplitude spectrum.

The first detection data processor 41 may not include the windowingprocessor 41 b and the Fourier transformer 41 c. In such a case, thetime domain waveform that is split into frames by the pre-processor 41 ais transmitted to the processor 10.

The first detector 40 may detect a shot sound and change the shot soundinto data. For example, in the case where the first detector 40 detectsa sound generated from a press machine, the shot sound is the soundgenerated by the pressing. In such a case, the pre-processor 41 aextracts an interval including the shot sound from the data detected bythe first detector 40.

FIG. 5A to FIG. 5D, FIG. 6A, and FIG. 6B are schematic views describingthe processing of the monitoring system according to the firstmodification of the first embodiment.

FIG. 5A shows a reference waveform. For example, the reference waveformand the length of time of a shot sound are pre-registered. FIG. 5B showsthe data transmitted from the first detector 40. In FIG. 5A and FIG. 5B,the horizontal axis is a time t; and the vertical axis is the magnitudeof the sound. The pre-processor 41 a calculates a cross-correlationcoefficient between the reference waveform shown in FIG. 5A and the datashown in FIG. 5B. FIG. 5C illustrates a cross-correlation coefficient cat each time t.

The value of the cross-correlation coefficient value is high at thetiming when a waveform similar to the reference waveform appears. Thepre-processor 41 a sets, as a starting point, a timing that exceeds aprescribed threshold and is when the cross-correlation coefficient has amaximum. From the data shown in FIG. 5B, the pre-processor 41 a extractsdata having a prescribed length of time from the starting point. FIG. 5Dshows the data extracted by the pre-processor 41 a.

Thereafter, similarly to the example described above, processing of theextracted data is performed by the windowing processor 41 b and theFourier transformer 41 c.

FIG. 6A shows the state of the second detector 50 when the first modelis generated. For example, the second detector 50 detects theacceleration of the three axes of the X-axis, the Y-axis, and theZ-axis. When the first model is generated, the X-axis and the Y-axis arealong the horizontal plane; and the Z-axis is aligned with the verticaldirection.

FIG. 6B shows the relationship between the time t and an acceleration Ain each axis direction. From a time t0 to a time t1, the acceleration inthe Z-axis direction is larger than the accelerations in the X-axisdirection and the Y-axis direction. The acceleration changes in eachaxis direction when the orientation of the first detector 40 changes atthe time t1.

For example, the second detection data processor 51 calculates theacceleration in each axis direction by using the signal transmitted fromthe second detector 50. The second detection data processor 51calculates an average value A1 from the latest acceleration V₀ to anacceleration V_(N) detected N previously. The second detection dataprocessor 51 calculates an average value A2 from an acceleration V₁detected one-previously to an acceleration V_(N+1) detected N+1previously. The second detection data processor 51 outputs thedifference between the average value A1 and the average value A2 to theprocessor 10.

The decider 14 compares the difference (the first value) between theaverage value A1 and the average value A2 to the prescribed secondthreshold. When the first value is not less than the second threshold,the decider 14 decides that the use of the first model is inappropriate.

The first detection data processor 41 includes, for example, aprocessing circuit. The first detection data processor 41 may beincluded in the first detector 40. Or, the processing by the firstdetection data processor 41 described above may be performed by theprocessor 10.

The second detection data processor 51 includes, for example, aprocessing circuit. The second detection data processor 51 may beincluded in the second detector 50. Or, the processing by the seconddetection data processor 51 described above may be performed by theprocessor 10.

The calculator 60 includes, for example, a processing circuit. Theprocessing by the calculator 60 may be performed by the processor 10.

FIG. 7 is a schematic view illustrating the configuration of amonitoring system according to a second modification of the firstembodiment.

In the monitoring system 120 according to the second modification shownin FIG. 7, the processor 10 further includes a corrector 15. When thecalculator 12 calculates the evaluation value, the calculator 12transmits the evaluation value to the corrector 15. The corrector 15corrects the evaluation value based on a previous evaluation value.

FIG. 8A to FIG. 8C and FIG. 9 are schematic views describing theprocessing of the monitoring system according to the second modificationof the first embodiment.

FIG. 8A to FIG. 8C and FIG. 9 are graphs showing the relationshipbetween the time t and an evaluation value E.

In the example of FIG. 8A, the evaluation value E increases as the timet elapses. For example, such a characteristic occurs due to thedegradation over time of the equipment. For example, the decider 14decides that the use of the first model is inappropriate at a time t2.The second model is generated by the calculator 60 and stored in themodel memory 30. In such a case, an evaluation value E0 that iscalculated at or after the time t2 is a value near zero.

In the example of FIG. 8A, an evaluation value El is a value less than afirst threshold th1 but greater than zero. In other words, compared towhen the condition of the equipment is good, the condition of theequipment is degraded when calculating the evaluation value E1. Forexample, even if the condition of the equipment is degraded whencalculating the evaluation value E1, the second model is generated usingthe first detection data when calculating. The degradation of thecondition of the equipment at or before the time t2 is not reflected inthe evaluation value based on the second model. As a result, there is apossibility that the evaluation value based on the second model is lessthan the first threshold th1 even when the original evaluation valuewould be not less than the first threshold th1.

For example, the calculator 12 calculates the second evaluation value byusing the second model and the first data. The corrector 15 corrects thesecond evaluation value. For example, the corrector 15 refers to thereference evaluation value when the second model is generated and thesecond evaluation value is calculated using the first data and thesecond model. The reference evaluation value is a previous evaluationvalue calculated before generating of the second model. In other words,the reference evaluation value is calculated using the first model andthe first detection data (the second data) transmitted to the processor10 before generating of the second model. For example, the referenceevaluation value is the evaluation value one-previous to the secondevaluation value.

The corrector 15 corrects the second evaluation value based on thereference evaluation value. For example, the corrector 15 adds thereference evaluation value to the calculated evaluation value. FIG. 8Bshows the result of this processing. The evaluation value E0 of FIG. 8Bshows the result of adding the reference evaluation value E1. By theprocessing of the corrector 15, the condition of the equipment based onthe evaluation value can be determined more accurately.

The corrector 15 may calculate an approximation formula for multipleprevious evaluation values including the reference evaluation value. Thecorrector 15 corrects the evaluation value by using the approximationformula. For example, as shown in FIG. 8C, the corrector 15 refers tothe previous evaluation values E1 to E5 when correcting the evaluationvalue E0. The corrector calculates an approximation formula APillustrating the relationship between the time t and the previousevaluation values E1 to E5. The corrector 15 corrects the evaluationvalue E0 at or after the time t2 by using the approximation formula AP.According to this processing, an evaluation value that reflects thecondition of the equipment more accurately can be obtained. Thecondition of the equipment based on the evaluation value can bedetermined more accurately.

An example is shown in FIG. 8C in which the approximation formula is alinear function. The approximation formula that is calculated by thecorrector 15 may be a function of second-order or higher-order. Forexample, as shown in FIG. 9, the approximation formula AP that is aquadratic function may be calculated based on multiple evaluationvalues.

Typically, as shown in FIG. 9, the degradation over time of thecondition of the equipment initially is gradual. It is known that whenthe degradation of the condition of the equipment subsequently starts,the degree of the degradation increases abruptly. By using theapproximation formula, the condition of the equipment can be determinedmore accurately based on the evaluation value even when the change ofthe condition is large.

FIG. 10 is a schematic view illustrating the configuration of amonitoring system according to the second embodiment.

The monitoring system 200 according to the second embodiment shown inFIG. 10 includes the processor 10. In the example shown in FIG. 10, themonitoring system 200 further includes the outputter 20, the modelmemory 30, the second detector 50, the second detection data processor51, the calculator 60, an imager 70, and an imaging data processor 71.

The imager 70 images the article which is the monitoring object andgenerates an image. The imager 70 includes, for example, a camera. Theimager 70 may include an infrared sensor. The imager 70 transmits agenerated image P to the processor 10. The image is, for example, acolor image. The image includes multiple pixels. For example, each pixelis represented by at least one of red, green, or blue. Each pixel may berepresented by a luminance signal, a differential signal of the bluecomponent, and a differential signal of the red component. Or, the imagemay be represented using a grayscale.

The imager 70 may transmit the image to the imaging data processor 71.For example, the imaging data processor 71 may cut out a part of theimage and perform processing as appropriate such as correctingdistortion, enlarging, reducing, rotating, grayscaling, binarizing, etc.The imaging data processor 71 transmits the processed image P to theprocessor 10.

The acceptor 11 accepts the image P. The calculator 12 calculates theevaluation value by using the image P and the first model stored in themodel memory 30. Similarly to the first embodiment, the first model is,for example, an autoencoder. The calculator 12 converts the image P intoone-dimensional array data and inputs the data to the first model. Aftercalculating the evaluation value, similarly to the monitoring system120, the determiner 13 determines the condition of the article based onthe evaluation value.

Similarly to the monitoring system 100, the decider 14 decides whetheror not the calculation of the evaluation value using the first model isappropriate based on the second detection data D2. When it is decidedthat the use of the first model is inappropriate, the calculator 60generates the second model that is different from the first model.

For example, the processor 10 accepts the first image output from theimager 70, and the first value output from the second detector 50. Theprocessor 10 performs the first determination when the first value is inthe first state. In the first determination, the processor 10 determinesthe condition of the article by using the first model and the firstimage. The processor 10 performs the second determination when the firstvalue is in the second state which is different from the first state. Inthe second determination, the processor 10 determines the condition ofthe article by using the second model and the first image.

Effects of the second embodiment will now be described.

When determining the condition of the article based on the image of thearticle, the state of the imager 70 affects the evaluation value. Forexample, the part of the article that is imaged changes when theposition or the orientation of the imager 70 changes. When the imagedpart changes, the image also changes; therefore, the evaluation valuealso changes. As a result, actually, there is a possibility that thecondition may be determined to be poor even when the condition of thearticle is good.

For this problem, there is a method in which the imaged image iscorrected based on a previous image. For example, there is a method inwhich the imaged image is corrected by causing a feature point insidethe imaged image and a feature point inside the previous image to match.However, in this method, the correction is difficult when there are fewfeatures such as shapes, colors, etc., in the surface of the article.For example, the method for correcting described above cannot be appliedwhen verifying scratches in a designated part of the surface of a metalplate, a semiconductor substrate, wood, etc.

There is also a method in which the correction is performed utilizingthe background other than the article inside the image. However, a partof the article may be imaged while enlarging to determine the conditionof the article with higher accuracy. Or, a part of the image that is cutout may be used in the determination. In such cases, the background maynot be included in the image.

In the monitoring system 200 according to the embodiment, the firstvalue that indicates the state of the imager 70 is output from thesecond detector 50. In the monitoring system 200, the model that is usedto calculate the evaluation value is decided according to the firstvalue. For example, as described above, the processor 10 performs thefirst determination when the first value is in the first state andperforms the second determination when the first value is in the secondstate. Thereby, the condition of the article can be determined moreaccurately even when the state of the imager 70 changes. The calculationamount that is necessary for the update of the model can be reduced.

For example, similarly to the monitoring system 100 according to thefirst embodiment, the processor 10 outputs at least one of thedetermination result, information indicating the update of the model,the evaluation value, or the first value. The processor 10 may outputthe image.

In the monitoring system 200, the second detector 50 may detect theenvironment where the article is provided. The processor 10 may acceptthe input of the first value indicating the state of the imager 70 orthe environment where the article is provided. The processor 10 mayaccept both the data indicating the state of the imager 70 and the dataindicating the environment where the article is provided. For example,the processor 10 accepts the first value indicating the state of theimager 70 and the second value indicating the environment where thearticle is provided. When the first value and the second value are inthe first state, the processor 10 performs the first determinationdetermining the condition of the article by using the first model andthe first data. When at least one of the first value or the second valueis in the second state, the processor 10 performs the seconddetermination determining the condition of the article by using thesecond model and the first data. According to this processing, thecondition of the article can be determined more accurately.

FIG. 11 is a schematic view illustrating the configuration of themonitoring system according to the embodiment.

For example, the monitoring systems according to the embodimentsdescribed above are realized using a computer 310 shown in FIG. 11. Thecomputer 310 includes a CPU (Central Processing Unit) 311, a inputdevice 312, a monitor 313, ROM (Read Only Memory) 314, RAM (RandomAccess Memory) 315, a storage 316, and a bus 317. The components areconnected by the bus 317.

The CPU 311 executes various processing in cooperation with variousprograms pre-stored in the ROM 314 or the storage 316 andcomprehensively controls the operations of the components included inthe computer 310. In the processing, the CPU 311 uses a prescribedregion of the RAM 315 as a work region. The CPU 311 realizes the inputdevice 312, the monitor 313, the communication device 317, etc., incooperation with programs pre-stored in the ROM 314 or the storage 316.

The input device 312 includes, for example, at least one of a keyboard,a microphone, or a touch panel. The input device 312 receives theinformation input from the user as an instruction signal and outputs theinstruction signal to the CPU 311. The monitor 313 includes, forexample, at least one of a monitor or a speaker. The monitor 313 outputsvarious information based on the signals output from the CPU 311. Themonitor 313 is one example of the outputter 20.

The ROM 314 non-reprogrammably stores programs used to control thecomputer 310, various setting information, etc. The RAM 315 is avolatile storage medium such as SDRAM (Synchronous Dynamic Random AccessMemory), etc. The RAM 315 functions as a work region of the CPU 311.Specifically, the RAM 315 functions as a buffer that temporarily storesvarious variables, parameters, etc., used by the computer 310, etc.

The storage 316 is a reprogrammable recording device such as a storagemedium using a semiconductor such as flash memory or the like, amagnetically or optically recordable storage medium, etc. The storage316 stores programs used to control the computer 310, various settinginformation, etc. The storage 316 functions as the model memory 30.

The embodiments may include the following aspects.

Aspect 1

A program, causing a processor to

-   -   accept first data output from a first detector detecting a        signal caused by equipment,    -   perform a first determination when a first value is in a first        state, the first value indicating a state of the first detector        or an environment where the equipment is provided, the first        determination determining a condition of the equipment by using        a first model and the first data, and    -   perform a second determination when the first value is in a        second state different from the first state, the second        determination determining the condition of the equipment by        using a second model and the first data, the second model being        different from the first model.

Aspect 2

A storage medium storing the program of Aspect 1.

According to the embodiments described above, a monitoring system and amonitoring method can be provided in which the condition of equipment oran article can be determined more accurately.

For example, the processing of the various data recited above isperformed based on a program (software). For example, the processing ofthe various information recited above is performed by a computer storingthe program and reading the program.

The processing of the various information recited above may be recordedin a magnetic disk (a flexible disk, a hard disk, etc.), an optical disk(CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, etc.), semiconductormemory, or another recording medium as a program that can be executed bya computer.

For example, the information that is recorded in the recording mediumcan be read by a computer (or an embedded system). The recording format(the storage format) of the recording medium is arbitrary. For example,the computer reads the program from the recording medium and causes aCPU to execute the instructions recited in the program based on theprogram. In the computer, the acquisition (or the reading) of theprogram may be performed via a network.

At least a part of the processing of the information recited above maybe performed by various software operating on a computer (or an embeddedsystem) based on a program installed in the computer from a recordingmedium. The software includes, for example, an OS (operating system),etc. The software may include, for example, middleware operating on anetwork, etc.

The recording medium according to the embodiments stores a program thatcan cause a computer to execute the processing of the variousinformation recited above. The recording medium according to theembodiments also includes a recording medium to which a program isdownloaded and stored using a LAN, the Internet, etc. The processingrecited above may be performed based on multiple recording media.

The computer according to the embodiments includes one or multipledevices (e.g., personal computers, etc.). The computer according to theembodiments may include multiple devices connected by a network.

Hereinabove, exemplary embodiments of the invention are described withreference to specific examples. However, the embodiments of theinvention are not limited to these specific examples. For example, oneskilled in the art may similarly practice the invention by appropriatelyselecting specific configurations of components included in monitoringsystems such as processors, outputters, model memories, first detectors,second detectors, calculators, imagers, etc., from known art. Suchpractice is included in the scope of the invention to the extent thatsimilar effects thereto are obtained.

Further, any two or more components of the specific examples may becombined within the extent of technical feasibility and are included inthe scope of the invention to the extent that the purport of theinvention is included.

Moreover, all monitoring systems, monitoring methods, programs, andstorage media practicable by an appropriate design modification by oneskilled in the art based on the monitoring systems, the monitoringmethods, the programs, and the storage media described above asembodiments of the invention also are within the scope of the inventionto the extent that the purport of the invention is included.

Various other variations and modifications can be conceived by thoseskilled in the art within the spirit of the invention, and it isunderstood that such variations and modifications are also encompassedwithin the scope of the invention.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the invention.

What is claimed is:
 1. A monitoring system comprising a processoraccepting first data output from a first detector detecting a signalcaused by equipment, the processor performing a first determination whena first value is in a first state, the first value indicating a state ofthe first detector or an environment where the equipment is provided,the first determination determining a condition of the equipment byusing a first model and the first data, the processor performing asecond determination when the first value is in a second state differentfrom the first state, the second determination determining the conditionof the equipment by using a second model and the first data, the secondmodel being different from the first model.
 2. The system according toclaim 1, wherein when the first value is in the first state, theprocessor outputs first information indicating the first model, and whenthe first value is in the second state, the processor outputs secondinformation indicating the second model.
 3. The system according toclaim 2, wherein the processor outputs the first value and one of thefirst information or the second information.
 4. The system according toclaim 3, wherein the processor further outputs the first data.
 5. Thesystem according to claim 1, wherein when the first value is in thefirst state, the processor outputs a first evaluation value indicatingthe condition of the equipment calculated using the first data and thefirst model, and when the first value is in the second state, theprocessor outputs a second evaluation value indicating the condition ofthe equipment calculated using the first data and the second model. 6.The system according to claim 5, wherein when the first value is in thefirst state, the processor determines, in the first determination, thecondition of the equipment based on the first evaluation value, and whenthe first value is in the second state, the processor determines, in thesecond determination, the condition of the equipment based on the secondevaluation value.
 7. The system according to claim 5, wherein when thefirst value is in the second state, the processor refers to a referenceevaluation value and corrects the second evaluation value based on thereference evaluation value, the reference evaluation value beingcalculated using second data and the first model, the second data beingoutput from the first detector before the second determination.
 8. Thesystem according to claim 1, wherein when the first value is in thesecond state, the processor generates, before the second determination,the second model by using third data output from the first detector. 9.The system according to claim 1, wherein when the first value is in thefirst state, the processor outputting a first determination resultindicating the condition of the equipment determined using the firstmodel and the first data, when the first value is in the second state,the processor outputting a second determination result indicating thecondition of the equipment determined using the second model and thefirst data.
 10. The system according to claim 1, wherein the firstdetector is a microphone or a vibration sensor.
 11. The system accordingto claim 1, wherein the first value is detected by a 3-axis accelerationsensor.
 12. The system according to claim 1, wherein the first statecorresponds to a first posture of an attached state of the firstdetector, and the second state corresponds to a second posture of theattached state to which the attached state of the first detector haschanged from the first posture.
 13. A monitoring system, comprising aprocessor accepting an input of a first image and a first value, thefirst image being output from an imager generating an image of anarticle, the first value indicating a state of the imager, when thefirst value is in a first state, the processor performing a firstdetermination determining a condition of the article by using a firstmodel and the first image, when the first value is in a second statedifferent from the first state, the processor performing a seconddetermination determining the condition of the article by using a secondmodel and the first image, the second model being different from thefirst model.
 14. A monitoring system comprising a processor acceptingfirst data output from a first detector detecting a sound or a vibrationcaused by equipment, the processor being configured to perform a firstdetermination of determining the equipment using a first data and afirst model when a posture of an attached state of the first detector isin a first state, outputting a result of the first determination, asecond determination of determining the equipment using a first data anda second model different from the first model when the posture of theattached state of the first detector is in a second state different fromthe first state, and outputting a result of the second determination.